In extreme classification settings, embedding-based neural network models are currently not competitive with sparse linear and tree-based methods in terms of accuracy. Most prior works attribute this poor performance to the low-dimensional bottleneck in embedding-based methods. In this paper, we demonstrate that theoretically there is no limitation to using low-dimensional embedding-based methods, and provide experimental evidence that overfitting is the root cause of the poor performance of embedding-based methods. These findings motivate us to investigate novel data augmentation and regularization techniques to mitigate overfitting. To this end, we propose GLaS, a new regularizer for embedding-based neural network approaches. It is a natural generalization from the graph Laplacian and spread-out regularizers, and empirically it addresses the drawback of each regularizer alone when applied to the extreme classification setup. With the proposed techniques, we attain or improve upon the state-of-the-art on most widely tested public extreme classification datasets with hundreds of thousands of labels.

In extreme classification settings, embedding-based neural network models are currently not competitive with sparse linear and tree-based methods in terms of accuracy. Most prior works attribute this poor performance to the low-dimensional bottleneck in embedding-based methods. In this paper, we demonstrate that theoretically there is no limitation to using low-dimensional embedding-based methods, and provide experimental evidence that overfitting is the root cause of the poor performance of embedding-based methods. These findings motivate us to investigate novel data augmentation and regularization techniques to mitigate overfitting. To this end, we propose GLaS, a new regularizer for embedding-based neural network approaches.

In the prior literature, they cited the low dimensional embedding methods is the reason of the poor performance of the embedding based methods. In this paper, the author proposed that the final score vector for the labels actually generated by highly non-linear transformation such as thresholding the scores. Thus it is not clear if the low-rank structure of the score vectors directly cause the low-rank on the label vectors. Furthermore, the author uses a simple neural network to mimic the low-dimensional embedding can attain near-perfect training accuracy but generalize poorly and suggesting that overfitting is the root cause of the poor performance of the embedding based methods. This is the first contribution of the paper which breaks the glass ceiling of embedding based methods.

There is some disagreement about the significance of the paper among the reviewers. Three steps can be distinguished. First, to refute the common belief that low-dimensional embeddings act as bottlenecks that limit the accuracy in the extreme classification case. Here, while it is true (raised by reviewer 1) that a representation result does not imply computational achievability, I feel that it reverses the direction of justification. If someone could show that common optimization methods fail to find embeddings (which "exist"), then this would re-instantiate the argument, yet in a more refined/precise form.

In extreme classification settings, embedding-based neural network models are currently not competitive with sparse linear and tree-based methods in terms of accuracy. Most prior works attribute this poor performance to the low-dimensional bottleneck in embedding-based methods. In this paper, we demonstrate that theoretically there is no limitation to using low-dimensional embedding-based methods, and provide experimental evidence that overfitting is the root cause of the poor performance of embedding-based methods. These findings motivate us to investigate novel data augmentation and regularization techniques to mitigate overfitting. To this end, we propose GLaS, a new regularizer for embedding-based neural network approaches.

LOS ANGELES, CA - FEBRUARY 06: Fei-Fei Li speaks onstage during The 2018 MAKERS Conference at ... [ ] NeueHouse Hollywood on February 6, 2018 in Los Angeles, California. After the COVID-19 pandemic is over and the economy reopens, many students will resume work on their careers. But for many young people, their priorities are going to shift. After seeing the pain and suffering caused by a single invisible enemy, some will naturally prioritize biomedical research over other easier and more lucrative trades, like law and finance. And some will choose to pursue possibly the most impactful area, which lies on the borderline of computer science and biomedicine - Artificial Intelligence (AI) for drug discovery.

In extreme classification settings, embedding-based neural network models are currently not competitive with sparse linear and tree-based methods in terms of accuracy. Most prior works attribute this poor performance to the low-dimensional bottleneck in embedding-based methods. In this paper, we demonstrate that theoretically there is no limitation to using low-dimensional embedding-based methods, and provide experimental evidence that overfitting is the root cause of the poor performance of embedding-based methods. These findings motivate us to investigate novel data augmentation and regularization techniques to mitigate overfitting. To this end, we propose GLaS, a new regularizer for embedding-based neural network approaches.

Recent developments in Named Entity Recognition (NER) have resulted in better and better models. However, is there a glass ceiling? Do we know which types of errors are still hard or even impossible to correct? In this paper, we present a detailed analysis of the types of errors in state-of-the-art machine learning (ML) methods. Our study reveals the weak and strong points of the Stanford, CMU, FLAIR, ELMO and BERT models, as well as their shared limitations. We also introduce new techniques for improving annotation, for training processes and for checking a model's quality and stability. Presented results are based on the CoNLL 2003 data set for the English language. A new enriched semantic annotation of errors for this data set and new diagnostic data sets are attached in the supplementary materials.

Last year, when Kate Brodock, CEO and founder of Women 2.0, pointed out that one of the key problems faced in the new tech industry the world over was that it was dominated by'white men'. Lack of racial, cultural diversity and most importantly, lack of gender equality was shaping the internet in a skewed manner. The scant number of women in new tech, especially in the areas of data science, analytics, and artificial intelligence has been a worrying trend in India as well. The resultant sexism is increasingly becoming one of the side-effects, making the global protests for gender equality so much more necessary. Anuradha Sharma, chief operating officer at Scienaptic says, "There is a very prominent male culture in programming and that kind of makes women less visible in data science and analytics field… Women in analytics are hidden, they are doing all the good work, but they are not asserting themselves. I know enough number of women in analytics, who are doing great work but they are not visible."

Here are choice articles on hot leadership topics culled from the business schools, the business press and major consulting firms, to start off your work week. Highlights include leading in the digital age, changing the game in industrial goods through digital services, the rise of machine learning, how women and men internalise the glass ceiling, and the explosion of wearing work on our wrists. Note: Some links require you to register or are to publications that have some form of limited paywall. "Servant leadership is not a new concept. Robert Greenleaf introduced the idea back in 1977. In recent years, however, concrete evidence has emerged that the approach delivers more than warm, fuzzy feelings. Last month, the first quantitative study that begins to explain a connection between servant leadership and improved individual performance was published by researchers in Canada. This new evidence may help move servant leadership from a niche practice to one adopted by more executives."